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The Use of Executive Control Processes in 
Engineering Design by Engineering Students 
and Professional Engineers 

Brophy, Klein, Portmore, and Rogers (2008) admitted that, as industries are 
driven by the rapid development of enabling technologies, they must become 
more flexible and adaptive to remain competitive. This flexibility is achieved 
through a workforce that can utilize newly available technologies and generate 
innovations of their own. They further suggested that such technological 
capability in the workforce can only be possible if students entering higher 
education are prepared differently at the K-12 level, through programs that 
target the development of technological literacy. 

Driven by the goal to improve technological literacy, the Standards for 
Technological Literacy: Content for the Study of Technology (ITEA, 2002) 
provide a framework for increasing students’ technological literacy at all levels 
of the K-12 curriculum through the integration of engineering design. In 
reference to the design component of the Standards for Technology Literacy, 
Lewis (2005) argued that it is "the single most important content area set forth in 
the standards, because it is a concept that situates the subject more completely 
within the domain of engineering” (p. 37). Consistent with its usage in society, 
engineering design provides an ideal platform for engineering and technology 
educators to integrate mathematics, science, and technology concepts for 
students to solve real-world (ill-structured) problems innovatively and 
creatively. 

Executive Control Processes 

A cognitive construct that is important when solving engineering design 
problems is executive control process, or metacognition. Fla veil (1978) and 
Brown (1978) define metacognition as knowledge and cognition about cognitive 
phenomena, or the monitoring of one’s own memory, comprehension, and other 
cognitive processes. Kellogg (1995) refers to metacognition as cognition about 
cognition, or thinking about thinking. It is a central feature of human 
consciousness that enables one "to be aware of, monitor, and control mental 
processes” (p. 211). 

In a synthesis of the literature on metacognition, Meijer, Veenman, and van 
Hout-Walters (2006) found that several studies identify some commonalities of 
higher order (executive control) cognition. For example, Schraw and Moshman 
(1995) subdivide metacognitive control processes into planning, monitoring, 
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and evaluation; while Pintrich and DeGroot (1990) divide metacognition into 
planning, monitoring, cognitive strategies, and awareness. O’Neil and Abedi 
(1996) also agree with the aforementioned researchers’ perception of 
metacognition; indicating that it includes planning, monitoring, and evaluation. 

Davidson, Deuser, and Sternberg (1995) discuss four metacognitive 
processes that are important contributors to problem-solving performance across 
a wide range of domains and problem types, including well-structured and ill- 
structured problems (see Figure 1). When a problem is given (including a design 
problem), the solver must decide what is known about the problem, what design 
criteria are expected, and what the constraints might be. They then use 
representations such as metaphors, analogies, and propositions to make sense of 
the problem and develop a solution. 

Metaphors and analogies are important representations used by designers in 
design problem solving (Casakin & Goldsmith, 1999; Daugherty & Mentzer, 
2008; Hey, Linsey, Agogino, &Wood, 2008). Metaphorical reasoning allows 
one to make conceptual leaps across domains from a source to a target, such that 
a new situation can be characterized and understood by reference to a similar 
one. In respect to designing, metaphors are often used in the early stages of the 
design process to assist the designer in framing the problem. Besides being used 
descriptively to define the problem and understand the situation, metaphors can 
also be used prescriptively as a solution generation tool. For example, “the 
metaphor, Shower Is A Reset, can be used to generate solutions that could 
support people’s feeling of starting anew even to the point of activating the 
shower with a button’’ (Hey et al., 2008, p. 288). 

An analogy can be defined as the “illustration of an idea by means of 
another idea that is similar or parallel to it in some significant features” (Hey et 
al., 2008, p. 283). Analogies make possible the solution of a problem in the 
target domain by superimposing upon it a solution from the base domain (Lewis, 
2008). In contrast to metaphors, analogies are generally used to solve functional 
issues and are used mainly during the generation of solutions, rather than in the 
framing of the design problem. 

According to Paivio (1990), propositions are the most versatile of 
representational concepts because they can be used to describe any type of 
information. They are strings of symbols that correspond to natural language and 
which “represent information in the same way regardless of whether the 
information is experienced verbally, as a spoken or written sentence in whatever 
language, or nonverbally, as a perceptual scene” (Paivio, 1990, p. 31). The 
relevance of propositions for engineering design lies in the fact that they can be 
expressed as general principles, rules-of-thumb, or heuristics; specific physical 
laws, such as those used in physics; or mathematical formulas (Greca & 

Moreira, 1997). 
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Figure 1 

Metacognitive processes in problem solving (Davidson et al., 1995). 



According to Davidson et al. (1995), planning entails dividing the problem 
into sub-problems and devising the sequence for how the sub-problems should 
be completed. Individuals are more likely to engage in planning when solving 
ill-structured problems because the situation is often novel and complex, so 
planning or structuring brings clarity to one’s intended actions. The plan is often 
revised or modified as the problem solver confronts obstacles during the 
solution process. This is consistent with Jonassen’s (1997) view that ill- 
structured problems possess multiple solutions because they can have multiple 
representations and multiple problem spaces. Research shows that individuals 
with less expertise in solving a particular type of problem spend less time in 
global “up front” planning, and relatively more time in attempting a solution, 
than do experts across age levels and areas of expertise (Davidson et al., 1995). 
Studies show that designers select features of the problem space to which they 
choose to attend and identify areas of the solution space they choose to explore 
(Cross, 2006). Junior engineering students tend to gather a lot of information 
when solving a design task, while more experienced designers ask for less 
information, process data instantly, and quickly build an image of the problem. 

While Davidson and associates (1995) assert that monitoring as an 
executive control process is concomitant with evaluation, some researchers, 
however, treat both as separate processes (see Flavell, 1979; Kincannon et al., 
1999; Schraw & Moshman, 1995; Veenman, van Hout-Wolters, & Afflerbach, 
2006). For the purpose of this study, both were treated as separate processes. 
Schraw and Moshman (1995) define monitoring as one’s awareness of 
comprehension and task performance, as well as the ability to engage in periodic 
self-testing while learning or solving a problem. The monitoring process relies 
on a variety of memories (such as idiosyncratic memories, emotional memories, 
and problem-related memories) and also on abstract rules. Although engineering 
design problems are ill-structured and contextually driven, the problem solver 
must apply abstract rules or propositions, like those used when solving well- 
structured problems in knowledge domains such as mathematics and physics, in 
order to achieve an optimal solution. 

Evaluation is the appraisal of the products and regulatory processes of 
problem solving. According to Schraw and Moshman (1995), this typically 
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includes re-evaluating one’s goals and conclusions. The representations used by 
problem solvers are referenced as they appraise their performance. Davidson et 
al. (1995) purport that evaluation includes control over the internal 
representations formed, and those that still need to be formed, for understanding 
and solving the problem. Jonassen (1997) further adds that evaluating one’s 
performance after the implementation of a solution includes the designer 
appraising: (a) whether the solution produced is acceptable to all the parties 
involved, (b) whether the solution is within the problem constraints articulated, 
(c) whether the solution is elegant or parsimonious, and (d) whether the effects 
of the solution could be optimized. 

Conceptual Framework 

The framework for this study was conceptualized by integrating the model 
for creative design, which illustrates the co-evolution of the problem and 
solution spaces during engineering design problem solving (see Dorst & Cross, 
2001; Maher, Poon, & Boulanger, 1996), with executive control processes such 
as planning, monitoring, and evaluation. According to Maher, Poon, and 
Boulanger (1996), whenever engineers are solving design problems, their 
problem and solution spaces co-evolve with an interchange of information 
between the two mental spaces. Dorst and Cross (2001) confirmed the accuracy 
of the Maher et al. model in a protocol study of nine experienced industrial 
designers whose designs were evaluated on overall quality, creativity, and a 
variety of other aspects. For simplicity, the co-evolution of the problem and 
solution spaces is illustrated in Figure 2 by the overlap of the two ellipses. 
Superimposing elements of the Davidson et al. (1995) metacognitive model on 
the problem and solution spaces of Maher et al. raises questions about how 
designers use executive control processes throughout their problem and solution 
spaces. 

Figure 2 

Conceptual Model 
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Purpose of the Study 

The purpose of this study was to investigate if there are differences in the 
cognitive process of engineering students and professional engineers as they use 
executive control processes (i.e., planning, monitoring, and evaluation) in the 
problem and solution spaces while solving an engineering design problem 
conceptually. The following research questions guided the study: 

1. In what ways do the executive control processes (planning, monitoring, 
and evaluation) of engineering students and professional engineers 
differ in their problem and solution spaces? 

2. How are propositions, analogies, and metaphors distributed throughout 
the use of executive control processes by engineering students and 
professional engineers? 

3. What is the overall design strategy of the professional engineers and 
engineering students? 


Method 

A qualitative comparison of novice and expert engineers was conducted. A 
purposeful sampling procedure was used to select the participants. According to 
Gall, Gall, & Borg (2007), in purposeful sampling the goal is to select cases that 
are likely to be “information rich" in respect to the purpose of the study. The 
executive control processes of a small group of mechanical engineering students 
were compared with a small group of professional mechanical engineers. 

Participants 

An email was sent inviting juniors and seniors in a four year mechanical 
engineering program at a Midwestern university to particate in the study. Six 
mechanical engineering students agreed to participate, three junior and three 
senior undergraduates. The four professional engineers were recommended by a 
former associate dean of a college of engineering, who is also a member of the 
American Society of Mechanical Engineers. Each professional engineer is 
recognised as an expert in mechanical engineering design. Except for one 
professional engineer, their individual number of years in the profession 
exceeded the minimum 10 years of experience it generally takes to achieve 
expertise in a particular domain (Phye, 1986). The small sample size is typical 
of verbal protocal studies (Jiang & Yen, 2009; Trickett & Trafion, 2006). 

The Design Task 

Each participant was given the same design problem for which to find a 
conceptual solution. Before the design task was administered, it was vetted by 
two professionals in the field, an Engineering Technology professor with over 
20 years teaching experience and a Mechanical Engineering professor with over 
10 years experience as a manufacturing consultant and over three years 
experience teaching manufacturing principles. This review helped ensure that 
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the design task was sufficiently ill-structured and of an appropriate difficulty 
level to engage the students and professional engineers. The final design task 
was then reviewed by a professor who teaches the senior design project course, 
and the task was pilot tested with a mechanical engineer with over 20 years 
experience (see Figure 3). 

Figure 3 

The Engineering Design Task 


THE DESIGN TASK 

The objective of this engineering design activity is to understand the cognitive process of 
engineering designers as they solve a design problem. Verbal Protocol Analysis will be used. 
This means that as you solve the problem you will be required to “think aloud” (say aloud) 
what you are thinking. If you stop speaking I will remind you to resume speaking aloud as you 
solve the problem. Please include all the notes and sketches of your solution on the sketch pads 
that are provided. 

Duration: 1 hour 

The Context 

Fonthill is a hilly terrain in the District of St. Mary with narrow tracks and virtually non¬ 
existent roads. This area also experiences high amounts of rainfall yearly. There are several 
communities like Fonthill on this mountainous tropical island. Because of the very poor state 
of the roads the most frequent mode of transportation are motorcycles. Motorcycles are used to 
take residents to and from work, market, and school. While the residents see this system of 
transportation as essential, the government has serious concerns about the safety of the riders 
and their passengers. The government therefore secured a loan to purchase a fleet of 
motorcycles that are specially built to handle these rugged terrains. These motorcycles will be 
leased as taxis to specially trained riders. 

The Design Problem 

The Honda CRF230 shown on the next page is a cross between a dill bike and a street bike. 
Modify the Honda CRF230 so that it is robust enough to handle repeated journeys through 
these mountainous terrains that are prone to a lot of rainfall annually. The average cost of a 
new car in this country is about US$25000.00 and the government expects that the cost of this 
motorcycle will not exceed one-third this cost. The motorcycle must also: 

• Be equipped with more cargo carrying capacity and at the same time make the rear 
seating (pillion) more comfortable. 

• Have an improved rack or a holding system for carrying packages, books, or a reasonable 
amount of groceries on the motorcycle. The rack must be non-metallic but of sufficient 
sturdiness to withstand a rugged terrain, occasional brushing against rocks, and a lot of 
rainfall. 

• Be capable of enough horsepower to climb sections of mountains with slopes of 30 
degrees, carrying the rider and the pillion passenger. 

• Have a device to prevent the theft of helmets from the motorcycle. 
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Procedure 

The design task was administered at a time and place convenient for each 
participant. Pencils, erasers, and sketchpads were provided, along with the 
instructions for the design task. Each participant was allowed approximately one 
hour to complete the design solution. A $25 gift card was given to each 
participant. Participants were required to produce only one conceptual design. 

Data were collected primarily through Verbal Protocol Analysis. The first 
stage of data collection, referred to as concurrent protocol, was carried out while 
the design problem was being solved. The second stage of data collection, 
referred to as retrospective protocol, was performed immediately after the 
problem was solved. 

Each participant had the choice of doing a verbalization practice session of 
about five minutes, thinking aloud as they solved a simple mathematical 
problem, to prepare them for the study. After they were comfortable with the 
thinking aloud process, the task was administered. The participants were 
encouraged to speak aloud whatever they were thinking as they solved the 
problem. Their think-aloud verbalizations were audio recorded. If the 
participants stopped talking, they were prompted or reminded to continue to 
speak aloud what they were thinking. 

After each participant completed the engineering design problem, an 
interview was conducted to clarify sections of the protocol and to allow the 
participant to explain the executive control processes that were applied. Like the 
concurrent protocol, the interviews were audio recorded. Their response to the 
reflective interview questions served as a supplementary data source to the 
concurrent protocols. A general interview guide format was used. According to 
Gall et al. (2007), with the general interview format, no set of standardized 
questions is written in advance because the order in which the topics are 
explored and the wordings of the questions are not predetermined. 

Data Analysis 

The audio recordings of the concurrent and retrospective protocols were 
transcribed. The transcribed protocols were then segmented into think-aloud 
utterances, divided into sentences, and coded. The quality of the sketches was 
not evaluated since the objective of the study was to examine the mental 
processes of the engineering students and the professional engineers while they 
solved the design task. The sketches and notes, however, acted as a reference to 
clarify some sections in the protocols. 

The purpose of segmenting is to break the transcribed verbal protocol text 
into units (or segments) that represent discrete thoughts and can be coded with a 
pre-defined coding scheme. Codes were provided for nine predefined constructs 
identified from the literature reviewed on metacognition, analogies, problem 
solving, and design (e.g., Casakin & Goldschmidt, 1999; Cross, 2006; Schraw & 
Moshman 1995; Hey et al., 2008). The codes were consistent with the constructs 
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described in the model for metacognitive processes in problem solving 
(Davidson et al., 1995) and the model for creative problem solving (Dorst & 
Cross, 2001; Maher et al., 1996). 

The constructs representing the participants’ mental representation were 
proposition, analogy, and metaphor. Those representing the participants’ 
executive control processes were planning, monitoring, and evaluation. The 
mental spaces describing the problem-solving episode were problem space, 
solution space, and overlapping space (see Table 1, next page). There were a 
total of 270 utterance segments (150 for the professional engineers and 120 for 
the engineering students). 

Reliability coding was conducted by having two persons code seven pages 
of one transcript (Miles & Huberman, 1994). A reliability kappa coefficient of 
0.76 was calculated for the first coding. All disagreements between coders were 
resolved through discussion. A second coding was done by both coders on the 
same number of pages of another transcript and a reliability kappa coefficient of 
0.9 was calculated. One coder then completed the coding of the remaining 
transcripts. 
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Table 1 

Constructs, Codes, and their Meaning 

Construct 

Code 

Meaning 

Propositions 

Prp 

Mathematical and engineering science formula and rule-of-thumb or 
heuristics used for example in analysis—e.g., F= mv 2 /r; "lowering the 
frame will lower the center of mass." 

Analogies 

Anl 

Comparing an idea with another idea that is similar in structural and 
relational features—e.g., comparing the surface texture of a leaf with 
the surface texture of a plate in a battery; 

Comparing two types of motorbikes 

Metaphors 

Mta 

Allows one to make conceptual leaps across domains from a source to 
a target so that a new situation can be characterized and understood by 
reference to a familiar one. They help to provide meaning to a design 
situation—e.g., viewing a gas station design problem as an oasis. 
Understanding a design situation by comparing an electronic book 
delivery design to a restaurant metaphor (Hey et al., 2008). 

Planning 

Pla 

Dividing the problem into sub-problems and strategizing how to reach 
a solution—e.g., Gathering data, prioritizing the requirements in 
design brief identifying constraints. 

Monitoring 

Mon 

Engaging in periodic self-testing and assessment of the quality of 
design as one progress to a solution—e.g., Performing analysis; 
testing the accuracy of a formula, calculation, or sketch for the 
accuracy of a clamping force. 

Evaluation 

Eva 

Appraising or judging whether the solution of a design meets 
constraints, costs, and all the demands of the stakeholder; judging 
quality of two or more design—e.g., Appraising whether one 
component is designed with the cheapest material that can guarantee 
the required strength and quality required by the customers. 

Problem 

space 

Prb-sp 

Includes design activities such as gathering information, defining the 
problem, identifying constraints, specifying evaluation criteria, and 
initially searching alternative solutions. 

Solution 

space 

Sol-sp 

Includes activities such as developing a solution, sketching, drawing, 
deciding between two alternatives, optimizing a selected solution, and 
determining specifications. 

Overlapping 

spaces 

Prb- 

Sol 

The mental space where information is interchange between problem 
and solution spaces. Involves consulting the design brief to make 
verification then returning to the solution or start a new solution. 
Activities include analysis and the selection of alternative solutions. 
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Results 

Executive Control Process Frequency and Characteristics 

As illustrated in Figure 4, the frequency of planning activities for both 
groups decreased, while the frequency of the monitoring and evaluation 
activities for both groups increased, as they progressed from the problem space 
to the solution space. The frequency of the professional engineers’ executive 
control processes was higher in the solution space (83) than the engineering 
students (59). Overall, the engineering students had a higher frequency of 
planning activities than the professional engineers. 

Figure 4 

Frequency Histograms Comparing Engineering Students and Professional 
Engineer Executive Processes 
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The engineering students showed major increases in the frequency of their 
monitoring activities in the problem space (6) and solution space (42). The 
professional engineers also showed major increases in their monitoring activity 
from the problem space (2) to the solution space (45); however, the engineering 
students displayed more monitoring activities in the overlapping space (24) than 
the professional engineers (10). The professional engineers did not show any 
signs of evaluation in the problem space and showed very little in the 
overlapping space (2), but the frequency of evaluation in the solution space 
increased significantly (35). The professional engineers used more executive 
control activities on average than the engineering students. 

Table 2 

Characteristics of Executive Control Processes 


Metacognitive 

Regulation 

Planning 


_ Characteristics _ 

Engineering Students _ Professional Engineers 

Spent more time planning. Spent less time planning 


Monitoring 


Used analogies to help in the 
framing and understanding the 
problem. “So what I am doing 
right now is trying to think of 
other road vehicles, their seating 
like for example four wheelers, 
their seating and the racks are 
much wider, so we could 
possibly make the rear a little 
wider by extending the frame... ” 
The majority of metal 
representations were exhibited 
during monitoring, and analogies 
were used more frequently than 
propositions. 

Safety seems to be the main 
factor that drives the assessment 
and optimization of the quality 
of a solution. “The exhaust I 
think might cause a problem 
with the rider. I think the more 
shielding would have to be 
implemented to prevent the rider 
or any cargo from burning. ” 


Planning strategies more driven by 
engineering science principles 
rather than analogical features. “So 
I lowered the center of gravity of 
the load and extended the wheel¬ 
base for stability. Okay I have an 
initial concept for moving 
forward. ” 

The majority of metal 
representations were exhibited 
during monitoring, and analogies 
were used more frequently than 
propositions. 

Most of the monitoring activities 
focus at improving the customer 
safety and comfort. “But this I 
mean to make the passenger more 
comfortable we’ve got to do a 
better job of seating ” 


Evaluation 


Spent less time on evaluation Spent more time on evaluation 
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Table 2 illustrates the main characteristics of the engineering students’ and 
professional engineers’ executive control processes. These characteristics were 
identified by themes that were common in the protocols for all four of the 
engineering students and three of the professional engineers. 

Mental Representations and Design Strategy 

The engineering students and the professional engineers used different 
amounts of propositions, metaphors, and analogies in their planning, monitoring, 
and evaluation. Only three of the engineering students (Don, Gus, and Len) and 
one professional engineer (Mac) used metaphors while they were planning. One 
engineering student (Len) and one professional engineer (Mac) used a metaphor 
while carrying out monitoring activities. 

The engineering students used only analogies in their evaluation, except for 
one (Hank) who used both analogies and propositions. In contrast, two of the 
four professional engineers used both analogies and propositions in their 
evaluation; one used only analogies, and one did not use any mental 
representation. Overall, most of the mental representations that were used by the 
engineering students and professional engineers occurred while they were 
monitoring their design solution. One engineering student (Hank) deviated from 
this pattern, using most of his mental representation during evaluation. The 
second highest number of mental representations was used during the planning 
of the engineering students and during the evaluation of the professional 
engineers. 

There were several differences and similarities in the engineering design 
strategy used by the engineering students and professional engineers. The 
professional engineers, on average, took a longer time to solve the design task 
than the engineering students (professional engineers 47.17 minutes; 
engineering students 30.17 minutes). The protocols revealed that some students 
and professional engineers showed a determination not to deviate from an early 
concept. This behavior is similar to findings indicating that experienced 
mechanical engineering designers and senior design students tend to attach to 
early solution ideas and concept (Ball, Evans, & Dennis, 1994; Ullman, 
Dietterich, & Stauffer, 1988). For example, the student Len, stuck with an ATV 
design idea from the beginning to the end of his design. 

LEN: (After about 2 minutes into his solution) ...and if you like a 
back seat like an ATV type it would be considerable more comfortable 
than having two people on one motorcycle. (About 25 minutes 
later)... o&ay for safety my original design is definitely safer because 
it’s two people sitting in an enclosed area and the bars here would be 
metal so they at more of a roll bar like on ATVs. 

The professional engineer Kirk showed a similar attachment to a concept 
that he had from the beginning of his solution. 
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KIRK: (After about 3 minute into his solution) ...my initial thought 
was some sort of an articulated vehicle that would be attached to the 
rear of the motorcycle that would cany the passenger and/or luggage 
and provide stability. (About 32 minutes later). ..my original concept 
for two rear wheels revolves around a rickshaw type concept where you 
would still essentially have four tires for the total vehicle. The rickshaw 
would provide a stable ride for the passenger to get out cany lots of 
load; it would be a really nice solution. 

The general design recommendation from both groups was a motorbike 
with a carriage compartment at the back; flatter, lower seats with a backrest; and 
broad wheels and locks to secure the helmets. There was remarkable similarity, 
and not much variance, in the alternative solutions of both the engineering 
students and the professional engineers. For example, both groups considered 
using a saddlebag in the center of the bike, a four wheel ATV type vehicle, a 
three wheel ATV type vehicle, a bike with a passenger carriage to the side, and a 
bike with a luggage carriage that is pulled from the back. 

Discussion and Conclusions 

The small purposeful sample used does not allow for generalized statements 
to be made about the mechanical engineering design process of professional 
engineers and engineering students. The findings from this qualitative study, 
however, confirm previous findings of other studies and provide useful insights 
about the executive control processes of student and professional mechanical 
engineer designers. 

Three conclusions were drawn from the findings. The first is that expert 
planning and monitoring is driven by propositions, while the novice planning 
and monitoring is influenced by analogical comparisons. One possible reason 
the students used more analogical comparisons in their planning and monitoring 
is because they were not familiar with the type of design problem, and so they 
drew upon similar types of design to aid them in defining the problem and 
finding solutions. In contrast, the professional engineers, because of their years 
of experience, could easily understand the nature of the problem and, therefore, 
relied more on engineering science formulas and heuristics in their planning and 
monitoring. 

The second conclusion is that mental representations are used mostly when 
the engineering student and professional engineers are monitoring their design 
solutions, and the professional engineers are more balanced than the students in 
their use of analogies and propositions. This conclusion is reflective of one of 
the themes identified by Jonassen, Strobel, and Beng Lee (2006) in a qualitative 
study of engineers in their natural working environment. They found that instead 
of relying on one form of representation, engineers use multiple forms of 
problem representation in their day-to-day practice. 
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The third conclusion is that evaluation plays a larger role in the solution 
space of professional engineers, while engineering students do more planning in 
the problem space. The decrease in planning activities and increase in 
monitoring and evaluation activities, as the designers move from the problem 
space to the solution space, were consistent with what Davidson et al. (1995) 
implied about metacognition in problem solving. The findings, however, 
indicate that the engineering students did more planning than the professional 
engineers. This conflicts with literature on metacognition in problem solving. 

For example, Davidson et al. (1995) stated that "individuals with less expertise 
in solving a particular problem seem to spend relatively less time in global ‘up 
front’ planning for solution, and relatively more time in attempting to implement 
a solution than do experts” (p. 218). Atman et al. (2007) also found that expert 
mechanical engineers spent twice as much time in problem scoping activities, 
such as problem definition and gathering information, which are elements of 
planning. The professional engineers in this study may not have needed to spend 
much time for planning due to their past experience, as planning may be so 
familiar to them that they simply move into articulating their thoughts about 
solutions. 

It is not surprising that the professional engineers used more monitoring and 
evaluation in the solution space. In fact, the literature on metacognition indicates 
that experts excel in these self-regulatory and appraisal skills. Experienced 
engineers were observed to make preliminary evaluations of their tentative 
decision, perform final evaluation, balance systems of benefits and tradeoffs, 
and use guidelines and rules-of-thumb when making decisions (Ahmed, 

Wallace, & Blessing, 2003; Crismond, 2007). The time spent in decision making 
is likely to be related to the time spent generating and evaluating solutions 
(Radcliffe & Lee, 1989). 

Implications 

The fact that the professional engineers used multiple forms of 
representations strengthens the suggestion of Jonassen and associates (2006) that 
design curriculum and pedagogy should not rely exclusively on algebra, 
calculus, and trigonometric formulas to represent problems, but students should 
be taught how to supplement these propositional representations with other 
alternative qualitative problem representations. 

The Standards for Technological Literacy: Content for the Study of 
Technology’ state, “as a part of learning how to apply design process students in 
grade 9-12 should... evaluate the design solution using conceptual, physical, and 
mathematical models at various interval of the design process in order to check 
for proper design and note areas where improvement is needed” (ITEA, 2002, p. 
123). Evaluation is recognized as a higher order cognitive skill at which experts 
excel. Therefore, design curriculum and teaching strategies should target the 
development of these skills. Engineering and technology students should be 
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taught how to use both quantitative and qualitative analytical methods to frame 
their strategy and monitor their design conceptualizations. In later stages of the 
design process, students can be taught how to determine the best alternative 
solutions by the conducting of scientific tests. This will also improve their 
evaluative skills. According to Crismond (2007), “Students can develop their 
own guidelines based on tests they conduct by formulating design rules-of- 
thumb. Design rules-of-thumb can strengthen the link between science and 
engineering design and amount to intermediate abstractions that link the 
concrete realities of a particular mechanism and product with relevant concepts 
and laws from engineering and the natural sciences” (p. 27). 

The increased evaluation activities by the professional engineers were 
evident primarily when they reflected on or reviewed their processes and 
solutions. Self-monitoring and evaluation are associated with higher levels of 
design quality (Crismond, 2007). Therefore, strategies used in grades 9-12 ETE 
classrooms should allow students to reflect on and critique their own and other’s 
design process and product. Crismond recommends that giving students practice 
at identifying others’ design strategies can make their design-oriented 
metacognition more accurate and automatic. 

References 

Ahmed, S., Wallace, K. M., & Blessing, L. (2003). Understanding the 

differences between how novice and experienced designers approach design 
tasks. Research in Engineering Design, 14, 1-11. 

Atman, C. J., Adams, R. S., Cardella, M. E., Turns, J., Mosborg, S., & Saleem, 

J. (2007). Engineering design processes: A comparison of students and 
expert practitioners. Journal of Engineering Education, 96(4), 359-379. 

Ball, L. J., Evans, J. St. B. T., & Dennis, I. (1994). Cognitive processes in 

engineering design: A longitudinal study. Ergonomics, 57(11), 1753-1786. 
Brophy, S., Klein, S., Portmore, M., & Rogers, C. (2008). Advancing 
engineering education in P-12 classroom. Journal of Engineering 
Education, 97(3), 369-387. 

Brown, A. L. (1978). Knowing when, where, and how to remember: A problem 
of metacognition. In R. Glaser (Eds.), Advances in instructional psychology 
(Vol. 1, pp. 77-165). Hillsdale, NJ: Erlbaum. 

Casakin, H., & Goldschmidt, G. (1999). Expertise and the use of visual analogy: 
Implications for design education. Design Studies, 20(2), 153-175. 
doi:10.1016/S0142-694X(98)00032-5 
Crismond, D. (2007). Contrasting strategies of beginning and informed 
designers. One representation of learning progression in engineering 
design. Retrieved from 

http:eres.li.umn.edu/eres/docs/20642/crismond_designstrategy_paper.pdf 
Cross, N. (2006). Designerly ways of knowing. London, England: Springer. 


- 87 - 



Journal of Technology Education Vol. 24 No. 1, Fall 2012 


Daugherty, J., & Mentzer, N. (2008). Analogical reasoning in the engineering 
design process and technology education applications. Journal of 
Technology’ Education, 19(2), 7-21. 

Davidson, J. E., Deuser, R., & Sternberg, R. J. (1995). The role of metacognition 
in problem solving. In J. Metcalfe & A. P. Shimamura (Ed.), 

Metacognition: Knowing about knowing (pp. 208-225). Cambridge, MA: 
MIT Press. 

Dorst, K., & Cross, N. (2001). Creativity in the design process: Co-evolution of 
problem-solution. Design Studies, 22(5), 425-437. doi: 10.1016/SO 142- 
694X(01)00009-6 

Flavell, J. H. (1978). Metacognitive development. In J. M. Scandura & C. J. 
Brainerd (Eds), Structural/process theories of complex human behavior. 
Alphen a.d. Rijn, The Netherlands: Sijthoff & Noordhof. 

Gall, M., Gall, J., & Borg, W. (2007). Educational research: An introduction 
(3rd ed.). Boston, MA: Pearson Education. 

Greca, I. M., & Moreira, M. A. (1997). The mental representations: Models, 
propositions and images used by college physics students regarding the 
concept of field. International Journal of Science Education, 19(6), 711- 
724. doi: 10.1080/0950069970190607 ' 

Hey, J., Linsey, J., Agogino, A. M., & Wood, K. L. (2008). Analogies and 
metaphors in creative design. International Journal of Engineering 
Education, 24(2), 283-294. 

International Technology Education Association. (2002). Standards for 
technological literacy: Content for the study of technology (2nd ed.). 
Alexandria, VA: ITEA. 

Jiang, H., & Yen, C. (2009). Protocol analysis in design research: A review. 
Retrieved from 

http://www.iasdr2009.org/ap/Papers/Orally%20Presented%20Papers/Desig 

n%20Method/Protocol%20Analysis%20in%20Design%20Research%20- 

%20a%20Review.pdf 

Jonassen, D. H. (1997). Instructional design model for well-structured and ill- 
structured problem-solving learning outcomes. Educational Technology 
Research and Development, 45(1), 65-94. 

Jonassen, D. H., Strobel, J., & Lee, C. B. (2006). Everyday problem solving in 
engineering: Lessons for engineering educators. Journal of Engineering 
Education, 95(2), 139-151. 

Kellogg, R. T. (1995). Cognitive Psychology’. London, England: Sage 
Publications. 

Kincannon, J., Gleber, C., & Kim, J. (1999, February). The effects of 

metacognitive training on performance and use of metacognitive skills in 
self-directed learning situations. Paper presented at the National 
Convention of the Association for Educational Communications and 
Technology, Houston, TX. 



Journal of Technology Education Vol. 24 No. 1, Fall 2012 


Lewis, T. (2008). Creativity in technology education: Providing children with 
glimpses of their inventive potential. International Journal of Technology 
and Design Education, 19(3), 255-258. 

Lewis, T. (2005). Coming to terms with engineering design as content. Journal 
of Technology Education, 19(3), 255-258. 

Maher, L., Poon, J., & Boulanger, S. (1996). Formalizing design exploration as 
co-evolution: A combine general approach. In J. S. Gero & F. Sudweeks 
(Ed.), Advances informal design method for CAD. London, England: 
Chapman and Flail. 

Meier, J., Veenman, M., & van Hout-Wolters, B. (2006). Metacognitive 
activities in text-studying and problem-solving: Development of a 
taxonomy. Educational Research and Education, 12(3), 209-237. 

Miles, M. B., & Fluberman, A. M. (1994). Qualitative data analysis: An 
expanded sourcebook (3rd ed.). London, England: Sage. 

O’Neil, H., & Abedi, J. (1996). Reliability and validity of a state metacognitive 
inventory: Potential for alternative assessment (CSE Tech. Rep. No. 469). 
Los Angeles: University of California, Center for the Study of 
Evaluation/National Center for Research on Evaluation, Standards, and 
Student Testing. 

Paivio, A. (1990). Mental representations. New York, NY: Oxford University 
Press. 

Phye, G. D. (1986). Practice and skilled classroom performance. In G. D. Phye 
& T. Andre (Eds.), Cognitive classroom learning: Understanding, thinking, 
and problem solving (pp. 141-168). Orlando, FL: Academic Press. 

Pintrich, P. R., & DeGroot, E. V. (1990). Motivational and self-regulated 
learning components of classroom academic performance. Journal of 
Educational Psychology, 82(1), 33-40. 

Radcliffe, D. F., & Lee, T. Y. (1989). Design methods used by undergrad 
engineering students. Design Studies, 10(4), 199-207. 

Schraw, G., & Moshman, D. (1995). Metacognitive theories. Educational 
Psychology’ Review, 7(4), 351-371. 

Trickett, S., & Trafton, J. G. (2006). A primer on verbal protocol analysis. 
Handbook of virtual environment training, 332-346. 

Ullman, D. G., Dietterich, T. G., & Stauffer, L. A. (1988). A model of the 
mechanical design process based on empirical data. A I in Engineering 
design and Manufacturing, 2(1), 33-52. 

Veenman, M. V. J., van Flout-Wolters, B., & Afflerbach, P. (2006). 

Metacognition and learning: Conceptual and methodological considerations. 
Metacognition and Learning, 1, 3-14. 


- 89 - 




